Aerial images of the earth's surface and its natural and man-made features taken from elevated platforms such as balloons, airplanes, and satellites are widely used for making maps, predicting weather, recording surface feature changes, identifying ground-based activities, and numerous other economic purposes. Invariably, images taken from an aerial camera, whether made using traditional photographic or electronic means, are not produced with alignment of the camera's photographic axis with the local vertical or rotational axis of the subject plane. Misalignment of the camera and object axes produces keystone distortion or a rotation of the recorded image. Such misalignment and rotation also occur for ordinary ground-based images taken from stationary or moving locations such as a picture of a building photographed from a location on a street, or of a land area photographed from an elevated location. Keystone distortion results in a geometric skewing of the location of the points or pixels in the captured image; or the picture may also be rotated in the image plane.
It is often important that points in the captured but distorted image be accurately located. For example, an accurate distance between two objects observable in the captured image may be required, which cannot be determined directly from measurements on a keystone-distorted image. In addition, when multiple images are sequenced and assembled to form a mosaic or a larger image than can be viewed with a single exposure, angular and rotational misalignments that produce distortion between adjacent images can result. Thus, the resulting recorded images can exhibit keystone and rotational distortions that must be resized by a geometric transformation of the captured images before they can be used for an end purpose.
FIG. 1 illustrates a photograph being taken of a rectangular plot of land 106 that lies on a “subject plane” with a rectangular grid 108. The photograph is taken from a camera 104 such as an elevated camera carried by an aircraft 102. The axis 110 of the camera is shown misaligned with the local vertical 112.
On FIG. 2, an exemplary resulting photograph 203 is illustrated that contains the “distorted image” of the rectangular plot of land lying on a “distorted image plane” and shows the rectangular plot of land as the general quadrilateral 206. The image of the rectangular grid 208 is correspondingly distorted, preventing direct measurement of distances that were recorded on the distorted image plane. To accommodate accurate measurements on a recorded image or to display a recorded image without keystoning or rotation, points on the distorted image plane must be mapped into a “corrected image” on a “corrected image plane” that restores the geometry and rotational alignment of the original subject. Thus, there are three “planes” that are discussed herein: a subject that lies on a “subject plane,” a distorted image that lies on a “distorted image plane,” and a corrected image that lies on a “corrected image plane.” The corrected image plane is ideally a scaled version of the subject plane.
A prior art method for correcting or accounting for keystone distortion for image processing systems is manual image correction, such as by physically moving the camera or other image acquisition device to align the optical axes. However, the system components may not be accessible for adjustment, or there may be a physical limitation on the placement of the imaging components that prevent sufficient adjustment to correct the distortion, such as in aerial photography. Another prior art method is to provide adjustable optical elements in the image processing system, such as special-purpose lenses, mirrors and mechanical arrangements that can correct keystone distortion. However, this approach may only be able to correct small distortions, and can be cost prohibitive or inconvenient to use because it relies on mechanical elements. Other, prior art methods for two-dimensional keystone correction are generally computationally intensive and may require a trigonometric transformation of the input data based on the angular misalignment of the camera and the subject plane, which may not be known at the time the image is corrected or processed. These computationally intensive approaches may also be cost prohibitive or otherwise impractical for many applications. This is particularly true for images electronically acquired, such as raster-scanned image, acquired by a Charge Coupled Device (CCD) camera or other electronic means that can produce multi-million pixels per image, and produce 60 or more images per second. Thus, what is needed in the art is a process for image keystone correction that can accurately locate points on a corrected image plane for recorded, processed, or displayed images that is not computationally intensive and does not require angular data that represents misalignment of the imaging components.